Papers with unlearning method
Balancing Forget Quality and Model Utility: A Reverse KL-Divergence Knowledge Distillation Approach for Better Unlearning in LLMs (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for unlearning large language models struggle with forget quality and model utility, leading to over-unlearning or partial unlearning. |
| Approach: | They propose a method that uses reverse KL-divergence based knowledge distillation for unlearning to achieve significant forget quality while maintaining model utility. |
| Outcome: | The proposed method outperforms existing methods in forget quality and model utility with larger unlearning datasets. |
Unlearning vs. Obfuscation: Are We Truly Removing Knowledge? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent methods often rely on obfuscation by injecting incorrect or irrelevant information to suppress knowledge, leaving models vulnerable to probing. |
| Approach: | They propose a method that flattens the model predictive distribution over automatically generated multiple-choice questions, effectively removing knowledge about target individuals. |
| Outcome: | The proposed method achieves unlearning with over 90% refusal rate and a higher uncertainty than obfuscation on probing questions. |
FaithUn: Toward Faithful Forgetting in Language Models by Investigating the Interconnectedness of Knowledge (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for unlearning undesirable knowledge have overlooked complexity and interconnectedness of knowledge, authors say . previous studies have neglected the complex nature of knowledge and neglected its internal dependencies. |
| Approach: | They propose a new concept called superficial unlearning to evaluate faithfulness of unlearning in knowledge QA settings. |
| Outcome: | The proposed method shows significant effectiveness in real-world knowledge QA settings. |
Rethinking Post-Unlearning Behavior of Large Vision-Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to remove knowledge from large vision-Language Models often fail to provide quality and informative post-unlearning responses. |
| Approach: | They propose a task that requires models to provide privacy-preserving yet informative responses for LVLMs. |
| Outcome: | The proposed method reduces the risk of unlearning after naive suppression by providing informative and visually grounded responses. |
Protecting Privacy Through Approximating Optimal Parameters for Sequence Unlearning in Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Language models (LMs) demonstrate exceptional capabilities on tasks, but are vulnerable to extraction attacks. |
| Approach: | They propose Privacy Protection via Optimal Parameters (POP) which induces the model to forget about some of its training data. |
| Outcome: | The proposed method outperforms the state-of-the-art in retaining LM performance on 9 classification and 4 dialogue benchmarks. |